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An  essential  component  of  simulation-based  military 
training  is  the  creation  of  an  opposing  force  that  is 
sufficiently  believable  within  the  context  of  the 
immersive  environment  and  provides  positive  training 
outcomes.  Constructive  systems  such  as  OneSAF  model 
the  doctrinally-expected  behaviors  of  the  opposing  forces 
so  that  soldiers  and  leaders  can  learn  to  anticipate  and 
counter  them.  Such  systems  work  well  when  enemy 
doctrine  is  relatively  fixed  and  well-understood,  but  do 
not  provide  positive  training  when  doctrine  -  if  it  can  be 
said  to  exist  at  all  —  is  fluid  and  not  well-understood. 

The  inherent  nature  of  asymmetric  warfare, 
particularly  terrorism  and  insurgency,  makes  the  value  of 
such  constructive  OPFORs  problematic.  In  missions  such 
as  peace-keeping,  occupation,  force  protection,  and  anti¬ 
terrorist  operations  in  Iraq,  Afghanistan  or  the 
Philippines,  the  enemy  is  unpredictable,  and  is  following 
an  unknown  and  constantly  shifting  set  of  rules  based  on 
values  frequently  not  well-understood  by  our  forces. 
Moreover,  the  enemy  is  often  difficult  to  distinguish  from 
a  local  civilian  population  that  is  in  itself  difficult  to 
understand.  Contact  with  the  enemy  may  be  indirect, 
through  analysis  of  transaction  data,  or  it  may  be  face-to- 
face.  Simulating  the  behaviors  of  such  a  wide  range  of 
potential  asymmetric  threats  only  through  programmatic 
methods  is  paradoxical  if  not  impossible. 

This  research  explores  the  use  of  human  role-players 
in  a  very-large-scale,  distributed,  persistent,  immersive 
virtual  environment  to  create  believable  opposing  forces 
and  civilians,  through  the  development  of  a  system  that 
would  enable  soldiers  and  their  leaders  to  practice  the 
challenges  inherent  to  asymmetric  and  other 
unconventional  missions.  These  challenges  include: 
developing  situational  awareness  based  on  appearance 
and  behavior,  analyzing  movement,  communications,  and 
transactions  over  time,  interpreting  foreign  languages, 
understanding  cultural  differences,  and  operational 
coordination  among  widely  distributed  groups,  including 
joint,  interagency  and  multinational  contexts. 

The  paper  describes  features  of  the  Asymmetric 
Warfare  -  Virtual  Training  Technology  (AW-VTT)  under 


development  by  the  U.S.  Army  RDECOM  STTC  and 
Forterra  Systems  Inc.  to  support  the  virtual  enactment  of 
such  complex  scenarios  as  urban  checkpoints  where 
coalition  forces  face  individuals  and  crowds  that  range 
from  complacent  to  belligerent,  and  convoy  missions  in 
insurgent-infested  areas  where  ambushes  are  common. 
The  AW-VTT  system  further  proposes  to  make  the 
employment  of  live  role-players  cost  effective  by 
enabling  the  participation  of  individual  users  equipped 
with  industry-standard  personal  computers  connected  to 
the  internet,  so  that  role-players  can  participate  from  any 
corner  of  the  world.  A  key  element  of  the  system  design 
is  the  creation  of  easy-to-use  drag-and-drop  scenario- 
creation  tools  to  enable  the  rapid  use  of  lessons-learned  in 
distributed  virtual  training  sessions. 

AW-VTT  SYSTEM  OVERVIEW 

The  AW-VTT  provides  large-scale,  persistent  3-D 
virtual  environment  that  allows  users  on  standard  PCs, 
distributed  over  the  Internet,  to  interact  in  real  time  at  the 
entity  level.  Those  users  can  engage  in  a  palpably 
immersive  exercise  of  blue  forces  interacting  with  a 
mixture  of  terrorist  OPFOR  and  civilian  populations. 
Each  user  is  represented  as  a  fully  animated,  3D  character 
(avatar)  which  models  culturally  appropriate  behaviors 
and  gestures.  Users  communicate  with  each  other  with 
voice,  text  chat,  and  instant  messaging,  which  enables 
face-to-face  interaction  as  well  as  simulation  of  a  range  of 
communications  technologies.  Integration  with  OneSAF 
Objective  System  entities  is  planned  to  enable  external 
control  of  entities  in  the  simulation.  In  addition,  APIs  are 
under  development  that  will  enable  autonomous  AI  agents 
to  control  avatars  in  real-time. 

REAL  HUMAN  INTERACTION 

Asymmetric  operations  and  military  operations  other- 
than-war  require  face-to-face  contact  with  civilians  and 
potential  terrorists.  State-of-the-art  military  simulators  do 
not  yet  provide  the  simulation  of  individual  characters  at 
the  entity  level  that  could  engage  a  trainee  in  anything 
resembling  a  human  dialog.  Constructive  simulated 
entities  are  purposefully  guided  by  programming  that 
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provides  a  repeatable,  controlled  learning  experience.  To 
realistically  simulate  an  asymmetric  threat,  the  entities 
must  not  only  display  believable  human  behavior,  but 
must  also  be  adaptable,  observing  the  tactics  employed  by 
the  trainees  and  modulating  the  response  to  them.  The 
AW-VTT  will  enable  live  role -players  to  interact  directly 
with  trainees  within  the  context  of  a  simulation,  using 
natural  human  speech  in  any  language,  and  to  vary  the 
interaction  as  needed  to  learn  the  tactics,  techniques  and 
procedures  (TTPs)  being  trained. 

The  AW-VTT  technology  is  centered  on  the 
immersive  experience  of  real  human  communication  in  a 
simulated  environment.  The  technology  enables 
participants  to  speak  while  piloting  their  avatars  and  to 
hear  each  other  based  upon  their  physical  location, 
whether  in  close  proximity  or  speaking  over  simulated 
communications  systems.  The  technology  synchronizes 
the  voluntary  and  involuntary  body  movements  of  the 
speaker’s  avatar,  including  stance,  gestures,  facial 
expressions,  breathing,  and  conversational  hand  waving. 
The  technology  provides  an  extensible  set  of  gestures  and 
facial  expressions  that  can  reflect  both  the  avatars’ 
cultural  background  (Arab  versus  Western)  and  state  of 
mind  (normal  versus  terrified).  Furthermore,  the  AW- 
VTT  provides  extensive  tools  for  customizing  avatars’ 
appearance  to  suit  their  roles:  complexions,  hair,  and 
clothing,  as  shown  in  Figure  1 . 


Fig.  1:  Changing  characters 


By  infusing  characters  with  believable  natural  human 
voices  and  speech,  believable  gestures,  and  suitable 
costumes,  trainers  can  design  scenarios  that  encompass 
the  realistic  challenges  of  addressing  men  and  women  that 
are  foreign  in  appearance  and  behavior,  in  a  wide  variety 
of  wartime  situations.  The  range  of  face-to-face 
encounters  possible  with  human-piloted  avatars  is 
unlimited.  By  also  equipping  role-players  with  terrorist 
paraphernalia  such  as  explosive  device  kits  that  can  be 
placed  and  detonated  in  a  wide  variety  of  remote 
locations,  the  AW-VTT  will  give  trainers  the  flexibility  to 
model  rapidly-changing  enemy  tactics. 

SIMULATED  HUMAN  CROWDS 

While  providing  the  ideal  level  of  believability  and 
expertise,  human  role-players  can  be  costly  to  deploy  in  a 
live  exercise.  Access  to  the  virtual  training  environment 


via  the  Internet  will  reduce  the  cost  of  their  participation 
by  not  requiring  role-players  to  be  physically  present  and 
outfitted  at  a  training  site.  Still,  the  labor  cost  and 
scheduling  logistics  of  live  actors  for  very  a  large-scale 
exercise  involving  thousands  of  entities  could  be 
prohibitive. 

Constructive  systems  such  as  the  OneSAF  Objective 
System  (OOS)  enable  modeling  and  control  of  behavior 
of  semi-automated  forces  above  the  individual  entity 
level,  to  control  units,  companies,  or  more.  A  significant 
part  of  this  research  will  be  to  link  the  OOS  to  AW-VTT, 
to  take  advantage  of  the  aggregate  entity  modeling  in  that 
system.  As  a  result,  the  AW-VTT  architecture  will  allow 
external  processes  in  lieu  of  humans  to  control  avatars. 
The  training  commander  will  then  have  the  flexibility  to 
deploy  live  role-players  only  when  they  are  essential.  For 
instance,  in  the  checkpoint  operation  depicted  in  Figure  2, 
the  drivers  of  vehicles  approaching  a  checkpoint  may 
need  to  be  able  to  converse  with  Soldiers  and  would  be 
live  role-players.  However,  the  characters  in  the 
background  -  the  passengers,  merchants,  people  walking 
in  the  distance  -  would  be  automated. 


Fig.  2:  Live  and  simulated  humans  at  a  checkpoint 


ADAPTIVE  BEHAVIOR 

A  technology  such  as  the  AW-VTT  enables  trainers 
and  trainees  to  rapidly  formulate,  enact,  adapt,  and  re¬ 
enact,  each  scenario.  The  technology  holds  the  potential 
of  enabling  role-players  to  both  play  out  doctrinally 
mandated  roles  and  to  perform  actions  that  reflect  recent 
experiences  in  the  field.  Unlike  game  “AI”  characters  that 
can  easily  be  recognized  as  fake  and  whose  behavior 
becomes  predictable,  live  role-players,  supported  by  a 
cast  of  simulated  humans,  can  present  the  level  of  surprise 
and  adaptability  that  is  currently  exhibited  the  enemy  in 
theaters  such  as  Iraq  and  Afghanistan.  Whether 
addressing  civilians  in  peace-keeping  operations  or 
battling  with  armed  combatant  terrorists,  rehearsing  these 
challenges  in  a  simulated  environment  will  help  to 
prepare  soldiers  for  the  unfamiliar  and  the  unexpected  in 
the  field. 
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